Rootkits are a stealthy type of malicious software (malware) designed to hide the existence of certain processes or programs from normal methods of detection. Rootkits typically enable continued privileged access to the compromised system. Rootkit installation may be automated or activated when an attacker obtains root or Administrator access. Obtaining this access may be either a result of direct attack on a system, exploiting a known vulnerability, or by getting access to a password (by cracking, privilege escalation, or social engineering). Once installed, a rootkit typically attempts to hide the intrusion, as well as to maintain privileged access for itself (or other processes).
Rootkit detection is difficult because a rootkit may be able to subvert the software that is intended to find it. Detection methods include: using an alternative, trusted operating system, behavioral-based methods, signature scanning, difference scanning, and memory dump analysis. Removal of rootkits can be complicated or practically impossible, especially in cases where the rootkit resides in the kernel. Reinstallation of the operating system may be the only available solution to the problem. When dealing with firmware rootkits, removal may require hardware replacement or specialized equipment.
Modern rootkits do not necessarily elevate access, but are often used to make another software payload undetectable by adding stealth capabilities. Most rootkits are classified as malware, because the payloads they are bundled with are malicious. For example, a payload might covertly steal user passwords, credit card information, computing resources, or conduct other unauthorized activities. A small number of rootkits may be considered utility applications by their users. For example, a rootkit might cloak a CD-ROM-emulation driver, allowing video game users to defeat anti-piracy measures that require insertion of the original installation media into a physical optical drive to verify that the software was purchased legitimately.
Rootkits can run at different privilege levels or modes of a computer environment. User-mode rootkits run at the same mode as most other user applications, rather than low-level system processes. They have a number of possible installation vectors to intercept and modify the standard behavior of application programming interfaces (APIs). Some inject a dynamically linked library (such as a .DLL file, .dylib file, a .so file, or a .shlib file) into other processes, and are thereby able to execute inside any target process to spoof it. Others with sufficient privileges simply overwrite the memory of a target application.
Kernel-mode rootkits (ring 0) and/or rootkits residing in the virtualization layer (sometimes referred to as ring −1) run with the highest operating system privileges by adding code or replacing portions of the core operating system, including both the kernel and associated device drivers. Most operating systems support kernel-mode device drivers, which execute with the same privileges as the operating system itself. As such, many kernel-mode rootkits are developed as device drivers or loadable modules, such as loadable kernel modules or device drivers. This class of rootkit has unrestricted security access. Kernel rootkits can be especially difficult to detect and remove because they operate at the same security level as the operating system itself, and are thus able to intercept or subvert the most trusted operating system operations and thereby “hide” themselves in a stealth-like manner. Rootkits and other malware that hide their existence on a system are difficult to detect and clean from within the scope of the infected operating environment.
Existing solutions that attempt to perform malware and anomaly detection using hardware event counters only contemplate using the time series data of specific counters, i.e., measuring and observing the rate of change in the counters over time. Such an approach has several drawbacks, e.g., the counters will reflect a mixture of events coming from different processes. Thus, it can only be successfully applied when a single workload is consuming a significant part of the system's computing resources. When several processes are running in parallel in any contemporary operating system (OS), it becomes highly problematic to discover the contributions of each particular process to the counters. Various techniques can be applied, e.g., expectation maximization (EM) algorithms, but they will also lack in details.
Thus, what is needed is a system that performs malware (and other anomaly) detection, leveraging both trusted hardware performance and event counters, as well as the addresses of the instructions that are generating the suspected malicious activity. By analyzing the address distribution's specific patterns, one can build a behavioral model (i.e., “fingerprint”) of a particular process—and later match malicious processes to the stored behavioral models whenever the actual measured behavior matches said stored behavioral models.